<!DOCTYPE html>
<html lang="en">

<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>How to Locate GCC Regressions</title>
<link rel="stylesheet" type="text/css" href="https://gcc.gnu.org/gcc.css" />
</head>

<body>

<h1>How to Locate GCC Regressions</h1>

<p>A regression is a bug that did not exist in a previous release.
Problem reports for GCC regressions have a very high priority, and we
make every effort to fix them before the next release.  Knowing which
change caused a regression is valuable information to the developer
who is fixing the problem, even if that patch merely exposed an existing
bug.</p>

<p>People who are familiar with building GCC but who don't have the
knowledge of GCC internals to fix bugs can help a lot by identifying
patches that caused regressions to occur.  The same techniques can be
used to identify the patch that unknowingly fixed a particular bug on
the mainline when that bug also exists as a regression on a release
branch, allowing someone to port the fix to the branch.</p>

<p>These instructions assume that you are already familiar with building
GCC on your platform.</p>

<h2>Search strategies</h2>

<p>If you've got sufficient disk space available, keep old install
tree around for use in finding small windows in which regressions
occur.  Some people who do this regularly add information to our
bug tracker about particular problem reports for regressions.</p>

<p>Before you start your search, verify that you can reproduce the
problem with GCC built from the current sources.  If not, the bug might
have been fixed, or it might not be relevant for your platform, or the
failure might only happen with particular options.  Next, verify that you
get the expected behavior for the start and end dates of the range.</p>

<p>The basic search strategy is to iterate through the following steps
while the range is too large to investigate by hand:</p>

<ul>
<li><a href="#get">Get GCC sources</a> for that date.</li>
<li><a href="#build">Build GCC</a>, or specific components that are
    needed for testing.</li>
<li><a href="#test">Run the test</a>.</li>
<li>Based on the outcome of the test, find the midpoint of the new
    search range.</li>
</ul>

<p>The first three steps are described below.  They can be automated,
as can the framework for the binary search.  The directory
<code>contrib/reghunt</code> in the GCC repository includes
scripts to do this work.</p>

<p>There are several <a href="#shortcuts">short cuts</a>
that can be used to shorten the elapsed time of the search.</p>

<p>Eventually you'll need to <a href="#identify">identify the patch</a>
and verify that it causes the behavior of the test to change.</p>

<p>There are a variety of problems you might encounter, but many of them
are simple to <a href="#problems">work around</a>.</p>

<h2 id="get">Get GCC sources</h2>

<h3>Check Out a Working Copy</h3>

<p>Check out a working copy using <a href="../git.html">Git</a>.
Use a new working copy that is separate from what you use for
development or other testing, since it is easy to end up with files in
strange states.</p>


<h3 id="dates">Branch and release dates</h3>

<p>If no one has provided a range of dates for when a particular
mainline regression appeared, you can narrow the search by knowing in
which release it first appeared and then testing the mainline between
the branchpoint for that release and the branchpoint for the previous
release that does not have the bug.  To find out the revision/date at
which a branch or tag was created, use the command <code>svn log
--stop-on-copy</code>.</p>


<h2 id="build">Build GCC</h2>

<p>The kind of bug you are investigating will determine what kind of
build is required for testing GCC on a particular date.  In almost
all cases you can do a simple <code>make</code> rather than <code>make
bootstrap</code>, provided that you start with a recent version of
<code>gcc</code> as the build compiler.  When building a full compiler,
enable only the language you'll need to test.  If you're testing a bug
in a library, you'll only need to build that library, provided you've
already got a compatible version of the compiler to test it with.  If
there are dependencies between components, or if you don't know which
component(s) affect the bug, you'll need to update and rebuild
everything for the language.</p>

<p>If you're chasing bugs that are known to be in <code>cc1plus</code>
you can do the following after a normal configure:</p>

<pre>
    cd <i>objdir</i>
    make all-build-libiberty || true
    make all-libiberty
    make all-libcpp || true
    make all-intl || true
    make all-libbanshee || true
    make configure-gcc || true
    cd gcc
    make cc1plus
</pre>

<p>This will build libiberty, libcpp, libbanshee, intl and <code>cc1plus</code>
(<code>make configure-gcc</code> is required since December 2002,
<code>make all-intl</code> since July 2003, <code>make all-libbanshee</code>
from May until September 2004, <code>make all-libcpp</code> since May 2004,
and <code>make all-build-libiberty</code> since September 2004).
Alternatively, you can do</p>

<pre>
    cd <i>objdir</i>
    make all-gcc TARGET-cc1plus
</pre>

<p>This works since October 2004.  When you have built <code>cc1plus</code>,
you can feed your source code snippet to it:</p>

<pre>
    cc1plus -quiet <i>testcase</i>.ii
</pre>

<h2 id="test">Run the test</h2>

<p>Assuming that there is a self-contained test for the bug, write a
small script to run it
and to report whether it passed or failed.  If you're automating your
search then the script should tell you whether the next compiler build
should use earlier or later GCC sources.</p>

<p>Hints for coming up with a self-contained test is beyond the scope
of this document.</p>

<h2 id="identify">Identify the patch</h2>

<p>The following commands are particularly useful to help you identify
changes from one version of a file to another:</p>

<ul>
<li><code>git log --help</code></li>
<li><code>git diff --help</code></li>
<li><code>git show --help</code></li>
<li><code>git annotate --help</code></li>
</ul>

<p>When you've identified the likely patch out of a set of patches
between the current low and high dates of the range, test a source tree
from just before or just after that patch was added and then add or
remove the patch by updating only the affected files.  You can do this by
identifying the revision of each file when the patch was added and then
using <code>svn update -r<i>rev</i> <i>file</i></code> to get the desired
version of each of those files.  Build and test to verify that this
patch changes the behavior of the test.</p>

<h2 id="shortcuts">Short cuts</h2>

<ul>

<li>If you've narrowed down the dates sufficiently, it might be faster to
give up on the binary search and start doing forward updates by small
increments and then incremental builds rather than full builds.  Whether
this is worthwhile depends on the relative time it takes to update the
sources, to do a build from scratch, and to do an incremental build.</li>

<li>Similarly, you can do incremental builds when going forward a small
amount from the previous build, and go back to builds in clean object
directories when building from earlier sources.  When moving forward and
doing incremental builds, use <code>contrib/gcc_update</code>.</li>

<li>Before building a compiler after updating the sources, check that
the new sources are different from the sources for the current ends of
the range.  If not, make this new date one of the endpoints without
doing the build and running the test.</li>
</ul>


<h2 id="problems">Work around problems</h2>

<ul>
<li>If one of the test builds fails, try a date or time slightly earlier or
later and see if that works.  Usually all files in a patch are checked in
at the same time, but if there was a gap you might have hit it.</li>

<li>Sometimes regressions are introduced during a period when bootstraps
are broken on the platform, particularly if that platform is not tested
regularly.  Your best bet here is to find out whether the regression
also occurs on a platform where bootstraps were working at that time.</li>

<li>If a regression occurs at the time of a large merge from a branch,
search the branch.</li>

<li>If a test causes the compiler or the compiled test program to hang,
run it from a <code>csh</code> or <code>tcsh</code> script using
<code>limit cputime <i>mm</i>:<i>ss</i></code> so it will fail if it
requires more than the amount of time you specified.  The same technique
can be used to limit other resources, including memory.</li>

<li>Latent bugs can become apparent due to to small changes in code sizes
or data layout.  Test failures for these bugs can be intermittent,
leading to randomness in a binary search for the patch that introduced
the bug.  This makes it important to see if the patch resulting from a
regression hunt looks as if it's actually related to the bug.  For
example, if a search on <code>i686-pc-linux-gnu</code> comes up with a
change to an unrelated target, you're probably looking for such a bug.</li>
</ul>

</body>
</html>
